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range of variables that influence learning, and whether three 
methods — content analysis, expert ratings, and meta-analys is — agree 
on whether and how strongly these variables influence learning, using 
the educational research literature and an expert survey. The 
presence of an emergent knowledge base for school learning was also 
investigated. A framework was developed to guide the study that 
incorporated the following theoretical constructs: (l) state and 
district governance and organization; (2) home and community 
educational contexts; (3) school demographics, culture, climate, 
policies, and practices; (4) design and delivery of curriculum and 
instruction; (5) classroom practices; and (6) student 
characteristics. Data from 179 handbook chapters (narrative reviews) 
were used for the content analysis, and a survey of 61 educational 
researchers provided expert ratings. Meta-analysis was conducted with 
91 studies. The magnitude of the correlation between the 
meta-analyses and the narrative reviews suggests an emergent 
knowledge base on school learning. The moderate correlation of expert 
ratings with the results from meta-analysis and narrative reviews 
also supports an emergent knowledge base on influences on learning. 
Three appendixes provide technical information on the narrative 
reviews and expert survey and a list of the 91 sources used in the 
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During the 1980s, two developments occurred that altered the way educators and researchers thought 
about teaching and learning; (1) a burst of educational reform which ushered in innovative programs and 
practices and (2) the creation of new research tools to evaluate the effectiveness of school interventions. 
The mediocre performance of the nation's students and the increased number of at-risk children and 
families spurred government, business, educators, and the public to rethink many aspects of schooling. 
Reform efforts included, for example, school restructuring, statewide academic standards, and the 
implementation of new assessment methods. Although many of these innovative practices have been in 
place for almost a decade, their long-term impact has not been established. As researchers begin to study 
the impact of these reforms, they have new tools to determine which programs, practices, and contexts 
produce the most robust and generalizable results. These tools are demonstrating that, in fact, some 
educational practices and programs work far better than others. 

This paper has three purposes: (1) to identify the relative effects of a wide range of variables that 
influence learning, (2) to determine whether three different methods of analysis— content analyses, expert 
ratings, and meta-analyses— agree on whether and how strongly these variables influence learning, and (3) 
to address the presence of a knowledge base underlying learning. 

THE 1980S: AN ERA OF SCHOOL REFORM 

In the United States, many sociological and educational trends have converged to create a climate of 
educational reform. With the release of A Nation at Risk (National Commission for Excellence in 
Education, 1983) and A Nation Prer^ared: Teachers for the 21st Century (Carnegie Forum on Education 
and the Economy, 1986), the public became aware of the mediocre performance of the nation’s schools 
and the inadequacies of the teaching corps. The poor performance of the United States’ students, when 
compared to their international counterparts (Stevenson, 1983; Stevenson, Lee, & Stigler, 1986), generated 
concern among business and civic leaders that the nation's graduates would be ill-equipped to deal with 
the demands of a global economy. 

Accompanying the glum reports on the nation's educational system, was news of an increase in the 
number of at-risk students. Demographic reports provided evidence of U.S. families beset by urgent 
problems including poverty, teenage pregnancies, single-parent households, substance abuse, limited 
health cate, and inadequate and unaffordable housing (Levy & Copple, 1989). These demographic trends 
point to the need for better education and social and health services in order to break the cycle of 
disadvantage facing at-risk students and their families (Schorr, 1989). 

Educators responded to the news of poor results and increasing numbers of at-risk children by 
restructuring schools, increasing standards for students and teachers, and developing a variety of 
innovative programs. The 1980s became an era of school reform during which most of the 50 states and 
many local districts participated in educational reform and improvement programs. 
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Many professional education organizations provided leadership in reforming the nation's schools. 
The National Governors Association, a pioneer in school restructuring, recommended fundamental 
changes in curriculum, instruction, and the patterns of decision making and accountability used in our 
nation's schools. Professional organizations for teachers, such as the National Science Teachers 
Association and the National Council of Teachers of Mathematics, issued new standards for students 
academic performance coupled with recommendations for assessing higher order thought processes, as 
well as factual recall. These same organizations also set standards for the performance of teachers 
(National Council of Teachers of Mathematics, 1989; National Science Teachers Association, 1989). 

Innovative programs and schools also demonstrated new directions for educational reform. The 
widely recognized programs include, for example: Comer's School Development Program (Comer. 
Haynes, & Hamilton-Lee, 1988), Levin’s Accelerated Schools Project (Levin. 1988), RJR Nabisco’s Next 
Century Schools (House Committee on Education and Labor, 1990), the Saturn School of Tomorrow 
(Norris, 1991), Sizer's Coalition of Essential Schools (Sizer, 1992) and Wang’s Adaptive Learning 
Environments Model (Wang & Toilers, 1990). 

The programs developed by Comer, Levin, Sizer, Wang and others have focused on serving at-risk 
student populations. New attitudes toward students with special needs, curricular innovations, and new 
instrucUonal strategies are among the key characteristics of these programs. New attitudes toward student 
diversity are reflected by recognizing students' prior knowledge, emphasizing strength-building rather 
than remediation, and distinguishing between cultural differences and deficits. Curricular innovations 
include focusing on complex, real-life problems; embedding basic skills instruction within these problems, 
and relating new content to students' prior knowledge and cultural background. New instructional 
strategies include modeling powerful thinking strategies, scaffolding complex tasks, employing reciprocal 
teaching, and using a variety of instructional approaches (Means & Knapp, 1991; Reynolds, 1989; Wang, 
Reynolds, & Walberg, 1986; Williams, Richmond, & Mason, 1986). 

Reports of students' poor performance also led to AMERICA 2000, the first federally initiated reform 
of U.S. schools (U.S. Department of Education, 1991). AMERICA 2QQ0; An Echicatipn Strategy was 
built around six national goals agreed upon by the 50 governors to increase the achievement of U.S. 
students. AMERICA 2000 has four tracks designed to promote educational excellence by improved 
accountability, new technology, lifelong learning, and greater parental and community involvement. The 
AMERICA 2000 program was adopted by all 50 states and implemented in thousands of communities 

across the United States. 

Professional education organizations; federal, state, and local governments; the business community, 
and the public all participated in the reform efforts of the 1980s. Although some of the new programs 
and practices of the 1980s were supported by research findings, few of these innovations have shovn 
replicable long-term impact on students. Kirst (1991) addressed the lack of long-term studies, saying. 
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"The paucity of longitudinal experiments and demonstrations has resulted in an overabundance of 
'snapshots,' studies of specific treatments and interventions, without a systematic knowledge base 
established over time and under varying circumstances" (p. 38). This paper is an attempt to begin to cull, 
from theory, empirical results, and expert judgements, a systematic knowledge base of school learning. 

NEW RESEARCH TOOLS: THE ADVENT OF META-ANALYSIS 

For nearly a century, educational researchers have examined the teaching and learning process 
(Walberg & Haertel, 1992). They have employed four basic types of research studies, including; (I) 
primary research analyzing original qualitative and quantitative data, (2) secondary analyses of original 
data, (3) narrative commentaries describing and critiquing studies without quantitatively summarizing 
them, and (4) research syntheses and meta-analyses quantitatively summarizing results of studies. 

Since the turn of the century, tens of thousands of primary research studies have been conducted 
focusing on teaching and learning. Traditionally, the results of these primary studies have been 
summarized in narrative reviews. Two examples of narrative reviews that have influenced school reform 
are What Works, a compendium of educational research results that was widely disseminated by the Uruted 
States Department of Education (1986), and the effective schools literature that featured lists of school 
characteristics associated with high academic performance (Brookover, 1979; Brookover & Lezotte 1977; 
Puiicey & Smith, 1983; Rutter, Maughan, Mortimore, Ouston, & Smith, 1979). 

Prior to the late 1970s, researchers lacked the statistical methods needed to conduct research 
syntheses and meta-analyses. Once the necessary statistical methods were developed, researchers were able 
to draw powerful conclusions based on dozens of comparable studies that were conducted over decades. 
Cook et al. (1992) describe the use of meta-analytic techniques for synthesizing particular literatures. 
They characterize meta-analysis as "...a set of quantitative techniques that permit synthesizing results of 
many types of research, including opinion surveys, correlational studies, experimental and quasi- 
experimental studies, and regression analyses probing causal models" (p. 4). Cook et al. (1992) provide 
examples of how meta-analysis can answer a wide range of research questions such as, why some marital 
therapies are more effective than others, why some intervention programs for Juvenile offenders achieve 
better results than others, and why gender differences in science achievement exist. 

Meta-analyses (or quantitative research syntheses) employed in the late 1970s and 1980s 
demonstrated the consistency of educational effects and placed teaching and learning on a more scientific 
basis (Gage, 1978; Glass, McGaw, & Smith, 1981; Walberg, 1986). Since the late 1970s, the number of 
meta-analyses and research syntheses has grown dramatically. Cook et al. (1992) cite Guzzo, Jackson, and 
Katzcll (1987), who reported that in the past decade the number of articles and reports indexed in 
PsychINFO under the term "meta-analysis" has grown steadily. In 1985 alone, nearly 100 meta-analyses 



were indexed. 
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Meta-analyses have been used to determine the effects of particular programs, contexts, and 
instructional practices on learning. Researchers, for example, quantitatively summarized the effects of 
school programs, such as cooperative learning and mastery learning (Johnson, Maruyama, Johnson, 
Nelson, & Skon, 1981; Guskey & Gates. 1986). Others synthesized the effects of particular learning 
contexts such as the home environment (Iverson & Walberg, 1982) and particular instructional practices, 
such as homework (Paschal, Weinstein, & Walberg, 1984). While each of these research syntheses and 
meta-analyses provided evidence of robust and generalizable findings concerning particular programs, 
contexts, or instructional practices, they did not provide information on the relative importance of the 
range of variables that impact learning. This paper compiles evidence about particular programs, contexts, 
and instructional practices, as well as other influences on school learning, and compares their relative 
influence. 



DEFINING A KNOWLEDGE BASE FOR SCHOOL LEARNING 

The educational challenges of the 1980s pointed out the need for a knowledge base for learning to 
guide reform. A knowledge base of school learning should include the learners' contexts, as well as the 
characteristics of learners themselves. It should not represent a particular philosophy, such as behaviorism 
or pragmatism. Rather, it should include theories explaining the influences on school learning, empirical 
results distilled from research studies, and expert judgments about influences on school learning. 

Psychological, sociocultural, and instructional theories of learning and schooling should be included 
in such a knowledge base. Empirical results in a knowledge base for school learning should contain 
several types of information, including the identification of specific variables affecting school learning 
and their relative influence. A wide range of variables would have to be considered, including student 
abilities, preferences, and prior achievement; teacher characteristics and classroom behaviors; instructional 
materials and practices; amount of time devoted to learning; curriculum content; classroom climate; 
characteristics of the school, home, and community; district and state educational policies; and 
demographic information characterizing students, schools, communities, and states. Experts would then 
be able to confirm or refute empirical findings based on their experience as practitioners and researchers. 

Theories, empirical results, and expert judgments are needed to establish a valid knowledge base. In 
this paper, then, the term knowledge base is used to represent the distillation of understandings from 
experts, narrative reviews and meta-analyses of variables that influence school learning. (The use of the 
term knowledge base, however, is not meant to imply the definition used in artificial intelligence research 
in cognitive science.) 
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DEVELOPING A THEORETICAL FRAMEWORK 

The first step in developing a theoretical framework was to identify a set of variables related to 
learning. The authors began this process by carefuUy examining several models of school learning, 
including those of Bennett (1978), Bloom (1976). Bruner (1966). Carroll (1963). Glaser (1976). and 
Hamischfeger and Wiley (1976). as analyzed within a systematic theoretical framework (Haertel. Walberg. 
& Weinstein 1983) These models contributed broad categories and specific variables to the theoreucal 
framework of school learning. For example, these models operationalized the category student ability, 
using variables such as aptitude, prior knowledge, verbal IQ. and pupil background. The category 
motivation was typically included in these models and was operauonalized as perseverance, self-concept of 
the learner, or attitudes toward school and subject matter. The category of classroom instruction was also 
considered important in all models and encompassed variables such as instructional events; clanty of 
instruction; and use of cues, feedback, and correctives (Wang. Haertel. & Walberg. 1990). 

The emerging theoretical framework was further extended by applying Walberg s (1980) nine factor 
model of educational productivity. Walberg’s model goes beyond the earlier models of school learning 
by emphasizing out of school influences and social-psychological variables. Walberg's nine factors are: 
student age or developmental level, ability (including prior achievement), motivation, quantity of 
instruction, quality of instruction, psychological environment of the classroom, influence of the home, 
influence of the peer group outside of school, and exposure to mass media. 

Also contributing to the emerging firework were the models of adaptive instrucuon (Wang. 1992; 
Wang & Walberg. 1985) that describe learning environments intended to maximize individual students 
opportunities for success in school. These models consider instructional delivery systems, program 
design, and implementation. They draw upon Glaser’s (1982) notion of "large practical variables.” which 
include "efficient allocation and use of teacher and student time, a classroom management system, 
systematic teacher feedback and reinforcement of student progress, instructional interactions based on 
diagnosed learning needs of individual students, and flexible administrative and organizational patterns 
responsive to program implementation and staffing needs” (Wang et al.. 1990. p. 31). 

Using effective schools literature (Edmonds. 1979). the authors also identified variables that were 
correlated with students' success in urban schools. Although school effectiveness has been defined in 
various ways (e.g., Austin. 1981; Brookover et al.. 1982; Kyle. 1985; Purkey & Smith. 1983; Rutter. 
1981). the research is remarkably consistent in identifying variables that are related to urban students 
successful academic performance. Based on the findings of Brookover (1979). Brookover and Lezotte 
(1977). Purkey & Smith (1983). and Rutter et al. (1979). the following characteristics of successful 
schools (both urban and non-urban) have been identified: curriculum articulation and organization, 
schoolwide staff development, parental involvement and support, schoolwide recognition of academic 
success, maximized learning time, district support, clear goals and high expectations, an orderly and 
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disciplined school environment, and the principal's leadership in attending to the quality of instruction. 
More recent studies identify new variables that are especially suitable for use in inner-city school 
interventions including careful recording of student progress, the principal's selective influencing of 
teaching strategies, and expressing high expectations for pupils' achievement (van De Grift, 1990). In 
summary, the effective schools literature suggests that changes in student and school-level performance 
are related to a variety of proximal variables such as instructional strategies and practices, as well as distal 
variables such as school restructuring, types of school organization, and state and local policies. 

All of these models of school learning contributed to the final theoretical framework by providing 
constructs and specific variables. In addition to the models of schooling reviewed above, selected sources 
were examined for potential specific variables. These sources included Brophy (1986); Segal. Chipman, 
and Glaser (1985): Glaser (1984); Keogh, Major-Kingsley, Omori-Gordon, and Reid (1982); Wang and 
Lindvall (1984); Wang, Reynolds, and Walberg (1987-91); and Wittrock (1986). 

From these sources and from the models of school learning, 224 variables were organized into a 
preliminary version of the theoretical framework. Members of the Scientific Advisory Panel of the 
Temple University Center for Research in Human Development and Education (including 12 prominent 
researchers in education) reviewed the theoretical framework and provided detailed commentaries. Using 
the panel members' suggestions, the authors added four more variables and revised the framework's 
organization. The final version of the theoretical framework contained 228 variables (a list of the 228 
variables is available from Margaret C. Wang, Temple University Center for Research in Human 
Development and Education, Ninth Floor, Ritter Hall Annex, 13th Street and Cecil B. Moore Avenue, 
Philadelphia, PA 19122), grouped into 30 categories, which were further organized within six theoretical 
constructs. 

The six theoretical constructs used to organize the framework, include: (1) State and District 
Governance and Organization; (2) Home and Community Educational Contexts; (3) School 
Demographics, Culture, Climate, Policies, and Practices; (4) Design and Delivery of Curriculum and 
Instruction; (5) Classroom Practices; and (6) Student Characteristics. Table 1 presents the six 
theoretical constructs and a description of the 30 categories that are classified within each of the six 
constructs. In addition. Table 1 presents the complete name of each category and an illustrative variable 
exemplifying each category. The following six sections describe why each theoretical construct was 
included in the framework, the types of research studies summarized within each construct, and the 
variables which operationalize that construct. 

State and District Governance an d Organization 

The theoretical construct of State and District Governance and Organization was included in the 
framework to refer to the effects of formal institutions of government on student learning and classroom 
practices. Educational policy in the United States is made in "1 10,000 schools, 15,000 school districts, and 
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thousands of state and federal agencies." (Grant, 1992, p. xii). Thus, government policy might be initiated 
at the school, district, state, or federal level (although federal policies are often transmitted through state 

and local agencies). 

For the purposes of this paper, the effects of policy developed at the school level are included within 
the theoretical construct School Demographics, Culture, Climate, Policies, and Practices. For example, 
effective schools research (Purkey & Smith, 1983), studies on school culture, and studies of organizational 
coherence would be considered within the School Demographics, Culture, Climate, Policies, and 
Practices construct. 

The effects of District Demographics and State and District Policies are categories within the construct 
State and District Governance and Organization. Studies of Chapter 1 staff development (Griffin, 
1986) and implementation of policies such as mainstreaming (Meisgeier, 1976) are also included. (See 
Table 1.) Federal policy, although not mentioned in the construct's title, is thought to be mediated 
through state and district policies. 

Because the U.S. educational system is not centralized the effects of federal government on schools 
have been weak, a product of both law and tradition (Cohen & Spillane, 1992). State governments have 
nominally controlled the U.S. educational system. Nevertheless, it has only been in the last few decades 
that state governments have actually exerted much power (Cantor, 1980). Local districts have traditionally 
exerted the most influence on U.S. schooling practices. This may account for the paucity of strong 
effects detected in studies of state and federal policy designed to promote school learning. 

Some examples of variables that operationalize this theoretical construct include policies for teacher 
licensure and guidelines for selection of curricula and textbooks, at the state level. Variables at the district 
level included teacher evaluation policies, per pupil expenditure, degree of school district 
bureaucratization, and presence of contractual limits on class size. 

Hottip and GommunitY Educational Contexts 

This construct, which includes four categories describing out of school influences on learning 
(Community, Peer Group, Home Environment and Parental Support, Student Use of Out of School Time) 
is part of the theoretical framework, because educators and researchers have long believed such contextual 
influences have strong effects on student learning. (See Table 1.) Although some researchers have 
questioned the strength of out of school influences, such as parent involvement, on learning (White, 
Taylor, & Moss, 1992), many other researchers have documented the benefits of family involvement in 
improving students' academic performance as well as enhancing improvements in school anendance. 
reducing numbers of dropouts, decreasing delinquency, and reducing pregnancy rates (Epstein. 1988. 
Graue, Weinstein, & Walberg, 1983; Moles, 1982; Peterson, 1989). 
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This construct includes not only family involvement but also community, home environment, and 
peer influences as well. Variables identified in research examining leisure time television viewing 
(Williams. Haertel, Haertel, & Walberg, 1982), socialization influences (Scott-Jones, 1974), home 
instruction and learning (Graue et al., 1983), and ethnocultural effects (Brantlinger & Guskin, 1987) were 
included in this construct. Only recently has the role of community as an influence on school learning 
been examined through empirical studies. Thus, the research base on community influences used in this 
paper is limited. 

Some community level variables which operationalize this construct include ethnic diversity and 
quality of social services for students. Examples of home environment and parental support variables 
include educational environment (e.g., number of books in the home), parental involvement in assuring 
regular school attendance, and parental interest in student school work. Peer group variables, for example, 
include level of peers academic and occupational aspirations, presence of a well-defined clique structure, 
and the absence of peer substance abuse and criminal activity. Other out of school variables, to name but 
a few. include student participation in extracurricular school activities, amount of time spent on homework 
and leisure reading. 

School Demographics. Culture. Climate. Policies, and Practices 

Since the early 1980s, both scholarly journals and the popular press have devoted considerable 
attention to the effects of schools on student achievement. The theoretical construct of School 
Demographics, Culture, Climate, Policies, and Practices includes the following categories: 
Teacher/Administrator Decision Making. School Culture, Schoolwide Policy and Organization, Parental 
Involvement Policy and School Demographics (See Table 1). Each of these is described below. 

School Culture is defined as an ethos affecting teaching and learning. Although out of school 
influences such as the socioeconomic status of the community impact school culture, such out of school 
influences are discussed in this paper under the theoretical construct of Home and Community 
Educational Contexts. Here, School Culture refers to the norms of schools, whether they reflea school 
efforts, community influences, or student characteristics. School norms can range from the status of 
athletics to attitudes toward graffiti or from the importance of making the honor roll to student respect 
toward teachers. Among researchers, the most commonly studied attributes of School Culture are 
variables such as staff retention, collaborative plaruiing and collegial relationships, use of cooperative goal 
structures, order and discipline, and the recognition of academic achievement (Turkey & Smith, 1983). 

The category Teacher/Administrator Decision Making reflects the degree of autonomy and 
importance that teachers and administrators have in addressing the needs of the school. Teachers 
sometimes play a decision making role in establishing schoolwide policies, such as uniform discipline, 
grouping practices, selection of curricula, allocation of resources, the role of team teaching, and effective 
u.sc of instructional time. Administrators' decision making has usually been viewed as having a greater 
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scientific vocabulary (Curbelo, 1985; El-Nemr, 1980; Shulman & Tamir, 1973). Curricular concerns like 
thematic units, use of multidisciplinary approaches, and use of culturally diverse materials have also been 
included in this category. 

Similarly, different modes of instruction produce different learning outcomes. White and Tisher 
(1986), for example, review the value of laboratory work versus tutorials, group study, self-study, and 
lectures as a means of training in problem solving. Results from a variety of primary studies indicate that 
students believe laboratory woiic produces different cognitive and affective benefits than teachers do. 
Instructional arrangements like mastery learning techniques, cooperative learning strategies, personalized 
instruction, and computer-assisted instruction are included in this category. 

The second category is comprised of Curriculum Design variables, which mediate curriculum 
through teacher presentation of material as well as texts and other educational media. Variables in this 
category, such as the use of advance organizers, affect student learning. For instance, eaiiy studies showed 
advance organizers had positive effects, but mixed results appeared in the 1970s. White and Tisher (1986) 
conclude that advance organizers are generally advantageous, though results are not conclusive. 
Examples of other curriculum design variables covered in this category include: employing specific 
objectives and learning hierarchies; linking assessment and diagnostic tests to curriculum; and making 
curriculum available to different size classroom groups and different kinds of learners. 

Program Demographics are included in the construct Design and Delivery of Curriculum and 
Instruction because variables such as very small class size have been found to influence student learning 
(Glass & Smith, 1979). Other demographic variables included in this category are: the number and size 
of instructional groups, the proportion of students with special needs served in regular classes, the number 
of classroom aides required, and the amount of curricular resources such as textbooks. 

Classroom Practices 

The theoretical construct Classroom Practices encompasses eight categories (See Table 1). This 
construct was included because teacher behaviors and classroom organization and management are linked 
to student outcomes (Brophy & Good, 1986; Rosenshine & Stevens, 1986). Each of the eight categories 
is discussed in further detail below. 

The category Classroom Implementation Support refers to variables that contribute to the 
implementation of an instructional program. These variables include creating and maintaining 
instructional materials, configuring classrooms to promote instruction, making use of classroom aides, 
using records to monitor student progress, establishing efficient classroom routines, communicating 
classroom rules and procedures, and developing student responsibility for independent self-study. 

Variables within the Classroom Instructional category reflect the research on the organization of 
instruction. It includes techniques to ensure students understand the goals of instruction and the content 
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being presented. Many of the variables in this category were identified through the research-based 
teacher effectiveness literature" including process-product research (Brophy & Good, 1986). Direct 
instruction is a clear example of the type of instructional technique included in this category. Other 
illustrative variables are rehearsal and elaboration of new concepts, sequencing of instructional events, 
confrontation of student misconceptions, signalling of transitions in lessons, redundancy in presentation of 
content, teacher enthusiasm about content, maintaining expectations about content mastery, providing 
frequent feedback, promoting metacognitive learning strategies, and scaffolding of instructior>. 

Variables within the Quantity of Instruction category have been well-researched and there is strong 
agreement that students need to be fully engaged in their academic pursuits and that teachers need to 
make wise use of instructional time (Berliner, 1979; Carroll, 1963; Hamischfeger & Wiley, 1976). 
Illustrative examples of variables in this category include; length of school year and day, time on task, 
time spent on educational activities such as classroom instruction, homework, field trips, and viewing 
educational television; and the nature of content missed in regular classrooms when students were removed 
for "pull out" programs. 

The category Qassroom Assessment includes standardized, curriculum-based, as well as teacher-made 
tests to measure student learning. This category is important because so much money has recently been 
allocated for local, state, and federal tests in the hope of improving student learning (CatteraU, 1990). 
Variables in this category include frequency of assessment, use of descriptive learner profiles rather than 
simple total scores, and assessment of higher order thought processes as well as basic skills in reading and 
mathematics. 

Classroom Management was included as a category in this theoretical construct because empirical 
findings abundantly demonstrate the effectiveness of particular classroom management techmques (Doyle, 
1986). Effective classroom management has been shown to increase student engagement, decrease 
disruptive behaviors, and enhance use of instructional time, all of which results in improved student 
achievement. Examples of variables in this category include minimal classroom disruptions, group 
alerting, learner accountability, transitions, and teacher "withimess." 

The category Student and Teacher Social Interactions refers to the frequency and quality of social 
interactions in the classroom. These interactions contribute to students’ sense of self-esteem and can foster 
a sense of membership in their class and school (Anderson, Everetson, & Brophy, 1979; Brophy & Good, 
1986). Among the variables included are positive verbal interactions among students and teachers, teacher 
reactions to students' answers, teacher use of praise, teacher coaching of appropriate social behavior, and 
student resjxinses to questions from students and teachers. 

A related category. Student and Teacher Academic Interactions, examines frequency and quality of 
academic interactions in the classroom. It encompasses teachers' questioning styles, praise, reinforcement, 
and use of correctives. Illustrative variables include the frequency and difficulty level of teacher questions 
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and student answers, the use of high post-question wait time, the frequency of calls for substantive oral and 
written responses. 

The category of Qassroom Qimate focuses on the socio-psychological dimensions of classroom life. 
Because classroom climate has a moderate positive influence on student achievement (Haertel, Walberg, & 
Haertel, 1981) it was included in this construct. Characteristics of a classroom with a positive climate 
include frequent and cooperative interactions between students and teachers, common interests and values, 
the pursuit of cooperative goals, a clear academic focus, weU-organized and well-planned lessons, explicit 
learning objectives, appropriate level of task difficulty for students, and an appropriate instructional pace 
(Haertel et al.. 1981). The variables in this category reflect these characteristics. 

Student C haracteristics 

This theoretical construct is composed of five psychological categories (Social and Behavioral, 
Motivational and Affective, Cognitive. Metacognitive, and Psychomotor). Also included is a Student 
Demographics category and a History of Educational Placement category (see Table 1). Educators have 
long acknowledged that school learning is strongly influenced by an individual’s psychological attributes 
or aptitudes (Como & Snow, 1986), demographic variables, such as gender (Becker, 1992), and students' 
history of educational placement (i.e., promotion, retention, or placement in special education). For this 
reason, all the above concerns were included in this theoretical construct Specific categories arc described 
further below. 

The first psychological category. Social and Behavioral variables, is important given the social nature 
of schooling. Many educators believe that children who engage in frequent disruptive behaviors, such as 
talking out of turn or hitting other children, often perform poorly in school, whereas cooperative children, 
who engage in positive and constructive behaviors, are more likely to perform well in school. Examples 
of social and behavioral variables are students’ positive, nondisruptive classroom behaviors, appropriate 
classroom activity levels, cooperativeness with teachers and peers, and ability to make friends. 

The variables in the Motivational and Affective category have received increased attention in the past 
decade, because students' interests, preferences, attitudes toward school and subject matter are recognized 
as important attributes that foreshadow their tendency to persevere and excel on school tasks. In the 
1970s, cognitive psychologists supplied new understandings of self-control and self-regulation as 
motivational orientations that support learning. Weiner (1976) and Lefcourt (1976) redefined 
"achievement motivation" and locus of control in terms of cognitive attributions, and Bandura (1977, 
1982) developed a cognitive theory of self-efficacy. While their contributions were not directly related to 
students' performance in school, more recent studies suggest that motivational and affective variables, long 
acknowledged as important by classroom teachers, must be considered as key attributes necessary for 
developing independent, self-regulated learners. Examples of variables illustrating this category are 
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attitude toward school, teacher, and subject matter; motivation for life-long learning; independence as a 
learner, perseverance on learning tasks; and academic self-competence tn subject area. 

The importance of variables in the Cognitive category is self-evident. Historically, tests of cognitive 
aptitudes are highly correlated with school achievement at all age ranges (Como & Snow, 1986). 
Cognitive aptitudes include general inteUigence, "nuid" and "crystallized” inteUigence, prior knowledge, 
prior competence in reading and mathematics, and verbal knowledge. Other examples of variables 
included in this category ai^ Piagetian stage of cognitive development; memory; levels of reasoning 
ability; and specific academic knowledge in subject areas. 

During the past decade, researchers have identified a variety of metacognitive processes and learning 
strategies that guide leamera as they perform complex tasks (Brown. 1978). These processes and learning 
strategies arc included within the Metacognitive category. Brown defines metacognitive skills as the 
planning, activating, monitoring, and evaluating of lower order cognitive skills. MetacognUive processes 
requirc mental awareness and self-appraisal of cognitive activities. Employing metacogniuve processes 
and learning strategies enhances students' academic performance (Weinstein & Mayer. 1986). The 
specific variables comprising the metacognitive category include self-regulatory and self-control 
strategies, comprehension monitoring, and positive strategies to cope with failure and to generalize 
concepts (Brown, 1980; Brown & Palincsar). 

The last psychological category. Psychomotor, has only one variable. Psychomotor skills specific to 
the arca being instructed were included in this theoretical construct, because they are relevant m any 
school learning task where physical dexterity may be important such as wnting. 

The Student Demographics category within the Student Characteristics constnict includes a range of 
variables such as chronological age, socioeconomic status, and gender which are correlated with school 
learning (Walberg, 1980). Variables such as ethnicity, first language, health status, and special education 
placement arc also related to school learning, and thus were included in this theoretical construct. 

The final category in the Student Characteristics constnict is History of Educational Placement. 
Promotion, rctention. or placement in special education are also related to school learning, justifying their 
inclusion in the theoretical construct 

METHODS 

The coUection. coding, transformation, summarization, and analysis of the data are desenbed below 
for each of the three methods of analysis-content analyses, expert ratings, and meta-analyses. Readers 
uninterested in the technical methodology may wish to skip to the Results section. Appendices 1 and 2 
provide specific technical details of the procedures used in collecting, coding, and aggregating the data 
used in the content analyses and the meta-analyses. Results from the Wang et al. (1990) content analyses 
of research literature, and Reynolds. Wang, and Walberg's (1992) survey of expert ratings are combined in 
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this paper with findings from 91 meta-analyses (quantitative syntheses). The uniqueness of this paper is in 
its comparison and contrast of results across the three methods of analysis and in the calculation of overall 
statistical estimates of effects. 

Content Analyses of Research Literature on School Learning 

Wang et al. (1990) reported content analyses of research literature on school learning. Data from the 
content analyses were employed in the current study. A synopsis of the corpus of studies, coding 
procedures, and the methods used to summarize the data are described below. A more detailed 
description is included in Appendix 1. 

Selection of a Corpus of Studies . From the large number of review articles on school learning, the 
authors chose 179 for the final corpus (a complete list of the bibliographic references for the 179 sources 
is found on pp. 38-42 of Wang et al., 1990). These comprised authoritative reviews and handbook 
articles, especially documents sponsored by the American Educational Research Association, government 
documents, and other resources. The final corpus also included studies recommended by the Scientific 
Advisory Board. (See Appendix 1.) The research literature included mostly sources describing K-12 
regular classroom learning but some articles were also included that concern the teaching and learning of 
mildly handicapped students. 

Coding Procedure. A three-point scale was used to code the strength of each of the 228 variables’ 
relation to school learning. Details of the coding procedure are described in Appendix 1. 

Data Summary . Means, standard deviations, alpha reliabilities, and frequency of mention were 
recorded for the 30 categories (Wang et al., 1990, p. 34). Appendix 1 provides more information about 
how the data was aggregated to represent each of the 30 categories. The alpha reliabilities reported for the 
30 categories ranged from .71 to .99; only four of the reliabilities were less than .80, and most exceeded 
.90. The means for the 30 categories reported in Wang et al. (1990) were used as data in this study. 

Expert Ratings of the Importance of Factors Related to School Learning 

Reynolds et al. (1992) reported results from a survey of educational researchers on the importance of 
specific variables to effective school learning. A description of the survey, its administration, and the 
method used for analyzing the data is summarized below. 

The Sample . The population was composed of 134 educational research experts who were first 
authors of the 179 major annual review and handbook chapters, book chapters, government documents, 
and journal review articles used in Wang et al. (1990). (Some authors wrote more than one chapter.) 
Each expert received a survey asking for ratings of the 228 variables along with a follow-up mailing. A 
total of 61 educational research experts responded for a return rate of 46%. 

The Survey . In addition to background information on the expert's area of specialization and 
institutional affiliation, the machine-scoreable survey asked for a rating on a four-point Likert scale of the 
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innuencc of each of the 228 variables on student learning. The scale ranged from "3." indicating strong 
influence on learning, "2," indicating moderate influence on learning, "1," indicating little or no influence 
on learning to "0," indicating uncertain influence on learning. The instructions defined learning to 
include both the process of learning and its outcomes. 

Data Summary . A mean rating across the 61 respondents was calculated for each of the 228 
variables. The means were aggregated into the 30 categories. 

Meta-Analvses of Research Literature on School Learning 

rnmiis of Studies Synthesized . Effect sizes and correlations were acquired from a six chapter special 
issue of the International Journal of Educational Research qJER) (Fraser, Walberg, Welch, & Hattie, 1987) 
which compiles the results of many meta-analyses of learning. See Appendix 2 for more information on 
the corpus of studies, included in Fraser et al. (1987). In addition, a library search was conducted to 
provide subsequent meta-analyses. Appendix 3 contains a list of the meta-analyses from Fraser et al. 
(1987) and the six additional research syntheses and meta-analyses that were included in the final corpus 
of studies. 

Prenaration of Data for Analysis . A set of decision rules was established to guide the selection of 
statistical results from the 12 tables of results presented in Fraser et al. (1987). These decision rules are 
presented in Appendix 2. The objective of these rules was to identify the quantitative results that best fit 
the 30 categories used in the theoretical framework in order to create a data set comprising a correlation 
or effect size for each of the 30 categories. Statistical results from the six additional meta-analyses 
acquired in the library search were also incorporated. Mean correlations or effect sizes were available for 
23 of the 30 categories. 

Some of the meta-analyses reported results in terms of effect sizes rather than correlations. These 
effect sizes were transformed into correlations (See Appendix 2.) There was one mean correlation for 
each meta-analysis used. Thus, for each of the 23 categories, there was a set of mean correlations. For 
example, the category Quantity of Instruction included results from three meta-analyses, and as such, 
comprised a set of three mean correlations. 

Next, a single weighted mean correlation for each of the 23 categories was calculated using the 
formula and procedure described in Appendix 2. This procedure was followed for each of the 23 
categories. 

Data Summary . A weighted mean correlation was recorded for 23 of the 30 categories. This set of 
23 mean correlations was used in this study. 

Analysis of the Content. Expert, and Meta-Analvses Data 

Analysis of the three sets of means (content analyses, expert ratings, and meta-analyses) included 
transforming the data, identifying outliers, and calculating summary statistics and correlations. Two of the 
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30 categories from the theoretical framework, Accessibility and History of Educational Placements, 
reflected variables commonly associated with special education practices and were eliminated for purposes 
of this analysis. 

Talculating Scaled Scores for Each Category and the Six T h eoreti cal Cpngtrycts 

In order for all three data sets to be in a comparable metric, the 28 mean ratings from the content 
analyses, the 28 mean ratings from the experts, and the 23 weighted mean correlations from the meta- 
analyses were transformed into z-scores. Z-scores are standardized scores with a mean of zero and a 
standaixl deviation of one. The z-scores were then transformed into T-scores which are scaled scores with 
a mean of 50 and a standard deviation of 10. This eliminated negative numbers and facilitated 
interpretation of the data. Using the T-scores, an overall mean was calculated for each of the six 
theoretical constructs and for each of the 28 categories. 

Several steps were completed to ensure the accuracy of data entry and transformation. The data were 
entered and verified. Hand checks were made on several of the transformations from effect size to 
correlation to ensure their accuracy, as well as on the transformations from raw data to z-scores and from 
z-scores to T-scores. All transformed scores were reviewed to confirm that the values were within the 
expected range. 

Identifying Outliers 

Also using the T-scores, least-squares linear regressions were computed between content ratings and 
expert ratings; between content ratings and meta-analyses; and between expert ratings and meta-analyses. 
Each of these three linear regressions was computed utilizing average T-scores for each of the 28 
categories in the content analysis and expert ratings, and for each of the 23 categories in the meta- 
analyses. Examination of scatterplots of residuals around regression lines was used to identify outliers, 
that is, discrepancies among the three sources of data. The residuals were also examined using the stem 
and leaf plots in Figure 1. 

Stem and leaf plots are a quick way to view a distribution of statistical data (Tukey, 1977). In stem 
and leaf plots, actual numerical values are presented. For each of the three sets of regression residuals in 
Figure 1, (content-expert, content-meta-analyses, expert-meta-analyses) a column, or stem, is presented. 
The left side of the stem contains the first digit of a residual. On the right side of the vertical stem is a row 
of digits, with each digit representing one leaf. Each leaf is the second digit of a residual. To read the 
stem and leaf plot, take the value to the left of the stem and combine it with each leaf to the right of the 
stem. For example, using the content-expert stem, there are four residuals presented with the values -.10, - 
.10, -.11, and -.15. There is also one residual with the value .15 and so on. Using these plots, residuals 
can be examined and outliers identified. For the content-expert residuals there is one clear outlier, .34. 
The same procedure was used to identify outliers for each of the three sets of residuals. Based on these 
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procedures, four categories were identifted as outliers. State Policy. Student Use of Out of School Time. 
Psychotnotor. and Program Demographies, and omitted from further correlational analyses. 

results 

n,e results of this study are presented in four sections. The first section characterizes the final corpus 
of 270 reviews (179 handbook chapters and reviews, and 91 meta-analyses). The second section presents 
an average T-score for each of the six theoretical constructs. Summary statistics, including average T- 
scores for the 28 categories are presented in the third section. In addition, the third section lists the highest 
and lowest T-scoies within each of the three methods (content analyses, expert ratings, and meta-analyses). 
TThe final and fourth section presents correlations between content analyses, expert ratings, and meta 

analyses. 

rharacteri furies of the Final Cor p us of 2 70 Review s 

Table 2 presenls the substamivc focus and disciplinary oriemation of the 270 reviews and syndieses 
thal were summarised in Ihe content analyses (179 handbook chapters and reviews) and in the 91 meta- 
analyses (research syntheses). The foci of Ihe reviews match Ihe six theoretical constructs used in 
developing the framewoik. The six foci range from indirect determinants of learning such as State and 
District Governance and Organisation to Ihe more direct detetminanls of student learning, including 
Design and Delivery of Curriculum and Instruction, Classroom Practices, and Student Characteristics. 
The academic disciplines teptesented in Table 2 include political science and policy studies, sociology and 
anthropology, psychology, and multidisciplinary studies, (i,e„ several social science disciplines, includmg 
some educational and curriculum studies). Psychologicai studies were further divided into three groups. 

(1) those studies that examined behavioral constructs such as reinforcement, cues, and token economies; 

(2) smdies examining cognitive constmets such as expectancies, anributions, metacognitive strategies, and 
teacher questioning styles; and (3) general psychological studies that encompassed both behavioral and 
cognitive constructs, or had no clear psychological orientation. 

The studies classified in Table 2 demonstrate that 70% of the education reviews and meta-analyses 
reported in this paper have a psychological orientation. 13% a multidisciplinary approach, 11% a political 
science or policy orientation, and 6% have a sociological or anthropological orientation. Classifying 
studies by focus and discipline shows that studies of state and district governance reflect a political science 
and policy perspective. Studies of home, community, and schoolwide contexts have a sociological and 
anthropological perspective. On the other hand, studies of more direct determinants of smdent learning, 
such as Design and Delivery of Curriculum and Instruction, Ciassroom Practices, and Student 
Characteristics, typically have a psychological perspective. 

The corpus of 270 reviews includes many studies of direct determinants of learning. Direct 
deteraiinants of learning are operationaliaed using proximal variables, those variables which have an 
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immediate effect on students. Student aptitudes and classroom practices are examples of proximal 
variables. Indirect determinants of learning are operationalized using distal variables which arc one or 
more steps removed from students' day-to-day lives. State and district policies and demographics arc 
examples of distal variables. Because fewer studies in this corpus examined distal variables, it is more 
difficult to generalize about their influence compared to the influence of proximal variables. 

All of the 270 reviews included in this corpus have student learning as a dependent variable. The 
majority of them operationalized student learning in terms of academic achievement. Examples of 
measures of these outcomes included; achievement tests, curriculum-based tests, and tests mandated by 
school districts, states, and the federal government. In some studies, variables can be related to learning on 
a single occasion whereas a different set of variables may be related to changes in learning over time. In 
this synthesis, it was estimated that 75% of the studies examined learning on a single occasion whereas 
25% examined learning over time. 

Average T-Scores for the Six TTieoretical Con.structs 

To understand better which of the six theoretical constructs most influenced student learning, the 
mean T-scores of all categories within each construct were averaged together, yielding a grand mean for 
each construct. Table 3 presents the grand means for each of the six theoretical constructs, oixlering them 
from greatest to least effect. 

In this research, the theoretical cortstmet with the greatest effect was Student Characteristics, followed 
by Classroom Practices, and Home and Community Educational Contexts. Having less effect were 
Design and Delivery of Curriculum and Instruction and School Demographics, Culture, Climate, 
Policies, and Practices, while State and District Governance and Organization had the least effect. 

Student Characteristics, Classroom Practices, and Home and Community Educational Contexts are 
direct determinants of student learning, while Design and Delivery of Curriculum and Instruction, School 
Demographics, Culture, Climate, Policies, and Practices, and State and District Governance and 
Organization are indirect determinants of student learning. Constructs that are close to students' defining 
characteristics and educative experiences, for example, psychological aptitudes, classroom practices, and 
home and community environments, exert the most influence on student learning. On the other hand, 
constructs that are removed from students' and their everyday learning experiences, like state and district 
policy, exert the least influence on student learning. 

Average T-Scores for the 28 Categories 

Table 4 presents the average T-scores for the three methods; content analyses, expert ratings, and 
meta-analyses, and an overall average T-score for each of the 28 categories. 

Average T'SCQrcs by Cstggory . The overall average T-scores for each of the 28 categories ranged 
from 64.8 to 32.9 (range = 31.9). The categories with the highest average T-scores carried the most 
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influence on student learning. The five most influential categories were: Classroom Management, 
Metacognitive, Cognitive, Home Environment, and Student and Teacher Social Interactions. The average 
T-scores for these five categories ranged from 64.8 to 56.7 (range = 8.1). Categories with the lowest 
average T-scores included: Program Demographics, School Demographics, State and District Policies, 
School Policy and Organization, and District Demographics. Average T-scores for these categories 
ranged from 42.8 to 32.9 (range = 9.9). These results demonstrated that proximal variables exert more 
influence than distal variables on school learning. The remaining 18 categories listed in Table 4 cannot 
be as easily characterized. However, many of the more influential categories were associated with 
psychological and classroom practice variables, both of which are proximal, while the less influential 
categories like out of school time and policies about parent involvement tend to be distal variables. 

Average T-Scores for Each of the Three Methods 

This section describes the range of T-scores for the content ratings, expert ratings, and meta-analyses 
(See Table 4). 

Content Ratings . Average T-scores for the content ratings of the 28 categories are presented in Table 
4. They ranged from 71.2 for the Psychomotor category to 22.4 for the State and District Policies 
category (range = 49.3). The categories with the five highest ratings included Psychomotor, 
Metacognitive, Classroom Management, Quantity of Instruction, and Student and Teacher Social 
Interactions, extending from 71.2 to 57.3 (range = 13.9). The Psychomotor category, which received an 
exceptionally high content rating, was considered an outlier when the correlational analyses were 
perfonned. Its high content rating was the result of two review articles that reported very strong effects 
based on Skinnerian and behavioral analyses of psychomotor skills. The five categories with the least 
influence were Parent Involvement Policy, Teacher and Administrator Decision Making, District 
Demographics, School Policy and Organization, and State and District Policies extending from 41.6 to 
22.4 (range = 19.2). Again, proximal categories, which include psychological and classroom variables, 
have more impact on learning than do distal categories which include policy and demographic variables. 

Expert Ratings . Average T-Scores for the expert ratings of the 28 categories are also presented in 
Table 4. The scones ranged from 68.0 for the Metacognitive category to 32.8 for the State and District 
Policies category (range = 35.2). The categories with the five highest ratings were Metacognitive, 
Classroom Management, Motivation and Affective, Home Environment and Parental Support, and 
Classroom Instructional, extending from 68.0 to 59.3 (range = 8.7). Categories receiving the lowest 
expert ratings were School Policy and Organization, Psychomotor, School Demographics, District 
Demographics, and State and District Policies, extending from 39.1 to 32.8 (range = 6.3). The 61 
educational researchers, who wrote definitive review articles and rated the importance of the 228 variables, 
clearly believed that proximal variables like psychological attributes, classroom instructional variables, and 
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the home environment, have the most influence on student learning, whereas distal variables like 
demographics and policy were rated as less important. 

Meta-Analvses . Average T-Scores for the meta-analyses, presented in Table 4, ranged from 70.2 for 
the Cognitive category to 32.6 for the Out of School Time category (range = 37.6). Unlike the other 
methods of analysis which included all 28 categories of variables, this method had only 23 categories of 
data available, because meta-analyses could not be found for five categories. The five categories with the 
highest average T-scores were Cognitive, Classroom Management, Home Environment and Parental 
Support, Metacognitive, and Student and Teacher Academic Interactions, extending from 70.2 to 59.3 
(range = 10.9). The five with the lowest ratings were School Policy and Organization, Classroom 
Implementation and Support, Psychomotor, Program Demographics and Out of School Time, extending 
from 40.8 to 32.6 (range = 8.2). Generally, the proximal variables included in the psychological, 
classroom instructional, and home environment categories had the most impact on school learning, while 
distal variables like policy and demographics once again had less influence. 

Degree of Consensus Among the Three Method .s of Analysis 

Pearson product-moment correlations were computed among content analyses ratings, expert ratings, 
and meta-analyses. Prior to computing each of these correlations, separate sets of oufliers were identified 
as described in a previous section (Identifying Outliers). 

Content Anaivses-Hxnert Rating Correlation . The correlation between content analyses and expert 
ratings is .704 (p < .01). Before computing this correlation, the following four categories were identified 
as outliers and not included in the analysis: State and District Policies, Out of School Time, Psychomotor, 
and Program Demographics. The correlation of .704 suggested that there is substantial agreement about 
what variables impact learning most, based on the agreement between experts' ratings and the content 
analyses of handbook chapters and narrative reviews. When asked to rate the 228 variables, the 61 experts, 
regaidless of their area of expertise, did not simply rate their own specializations as important, but instead 
rated the importance of variables based on a "transdisciplinary" understanding of what influences leanung. 

Content Analvses-Meta-Analvses Correlation . The correlation between content analyses and meta- 
analyses was .514 (p < .05), showing a moderate degree of agreement. The categories identified as 
outliers and eliminated before the correlation was computed were State and District Policies, Out of School 
Time, Psychomotor, and Program Demographics. This correlation provided evidence of "inter-source" 
reliability. The agreement between the meta-analyses and content analyses demonstrates the robustness 
and consistency of the results regardless of the source, whether it is a narrative review as represented in the 
content analyses or a meta-analysis. 

Expert Ratings-Meta-Analvses Correlation . The correlation between expert ratings and meta-analyses 
was .587 (p < .01), evidence of moderate agreement. Only one category. State and District Policies, was 
identified as an outlier and removed before the correlation was computed. This correlation demonstrates 
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agreement between expert ratings and the meu-analyses regarding the strength of different categories' 
effect on student learning. Thus experts' understanding of what impacts learning agrees with empirical 
findings established through 91 meta-analyses. This finding along with the other two correlations 
suggests general agreement among experts and empirically-based findings about what variables impact 
school learning and their relative strength. 

DISCUSSION 

Evidence suggests that an emergent knowledge base, though neither formalized nor explicit, underlies 
learning. This evidence comes from the disciplines of psychology, sociology, anthropology, political 
science, and multidisciplinary (including curriculum) studies. Because these different academic 
disciplines have directed their attention to different influences on schooling, they have helped to establish 
a knowledge base that looks at learning through many lenses, and that spans an array of influences on 
learning from the proximal to the distal. The presence of this knowledge base is demonstrated by the 
consensus of experts and findings from empirical research. Regardless of which method of analysis 
(content analyses, expert ratings, or meta-analyses) was employed, there was moderate to substantial 
agreement on the relative sizes of influences on school learning. 

Perspectives of Different Academic Disciplines 

The perspectives of different academic disciplines have directed researchers' attention to different 
types of variables in the study of learning. Although disciplinary views of education cannot be 
comprehensively summarized here, it may be useful to remind readers of a few characteristic contributions 
of these disciplines. Political scientists have focused on federal, state, and district level policy variables. 
Historically, sociologists have been concerned with demographic variables such as social class and 
minority group membership. Sociologists have contributed to the research on effective schools and have 
frequently advocated schoolwide organizational solutions to educational ills. Psychologists have directed 
their attention to the psychological and individual characteristics of teachers and learners and have 
attended not only to psychological characteristics of the learner, but features of the classroom, home, and 
community that foster learning and promote responsibility and independence in students. Diverse 
disciplines have provided educators with information on many types of variables, both proximal and distal, 
that comprise the knowledge base underlying academic learning. 

Correlations Support a Knowledge Base 

The handbook chapters and narrative reviews summarized in the content analyses often describe 
research results without quantifying them, nor do they cover comprehensively all primary studies or 
employ explicit search criteria in many cases. Meta-analyses, on the other hand, statistically summarize 
results of many primary studies and use explicit criteria for the inclusion and exclusion of studies. Thus, 
results from the handbook chapters and narrative reviews are not isomorphic with the meta-analyscs. 
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Nevertheless, the magnitude of the correlation between the meta-analyses and the narrative reviews 
suggests an emergent knowledge base. The moderate correlation of expert ratings with results from both 
the meta-analyses and the narrative reviews further suggests an emergent knowledge base on influences on 
learning. 

The Relative Importance of Distal and Proximal Variables 

Distal variables, like state, district, and school level policy and demographics, have little influence on 
school learning. This finding is inconsistent with current conventional wisdom which argues for policy- 
driven solutions, like school restructuring, school-site management, and tougher teacher credential 
requirements and evaluation, to improve student learning. Characteristics of effective schools have been 
documented to some extent (Holmes, 1989). Consistent with the findings of this paper, however, recent 
research is providing evidence of the limited impact of some schoolwide policies, such as special education 
classification and placement, (Gamoran & Berends, 1987; Wang, Reynolds, & Walberg, 1988) tracking, 
(Oakes, 1985) and retention (Holmes, 1990; Shepard & Smith, 1989) on student outcomes. The moderate 
degree of consensus across the three methods of analysis illustrates the common understanding that distal 
variables have lesser impact. This understanding also contributes to the knowledge base on learning. 

Distal variable are at least one step removed from the daily learning experiences of most students. 
Simply instituting new policies, whether state, district, or school level, will not necessarily enhance student 
learning. Implementing a policy of maximized learning time, for example, does not guarantee that 
students in a given classroom will receive instruction from a teacher who plans lessons with special 
attention to eliminating poor management practices and inefficient use of time. Policies do not always 
reach down to the classroom level. Effective policies require implementation by teachers at the classroom 
and student level. 

In a recent interview, Cohen (Brandt, 1991) described the work of the National Alliance for 
Restructuring Education. The organization's efforts to assist in school restructuring for a network of five 
states and seven districts provides further evidence of the limitations of distal variables and the importance 
of proximal variables in improving student outcomes. Cohen characterizes proximal variables as equally 
if not more important than distal variables in tackling school problems. He further maintains that schools 
should begin solving problems by addressing proximal variables like curriculum, instruction, and 
assessment that emphasize student outcomes. 

Based on the results of content analyses, expert ratings, and meta-analyses summarized in this paper, 
proximal variables strongly influence school learning. Proximal variables like psychological, 
instructional, and home environment variables have more impact on learning than most of the variables 
studied and should be part of an effective strategy to promote student learning. 
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Key Proximal Variables Influence Student Learning 

Key types of proximal variables — psychological, instructional, home environment -- which exen 
especially strong effects, are described below. 

P<;vchological . The psychological aptitudes which play the most significant role in school learning 
are metacognitive, cognitive, motivational and affective variables; each is discussed further. One of the 
most significant educational findings of the last decade has been the documentation of metacognitive 
processes that serve to guide students through learning tasks. Many research articles have described 
metacognitive processes and applications such as comprehension monitoring, strategies to facilitate 
generalization of concepts, self-regulatory and self-control strategies, cognitive skills instruction, and 
reciprocal teaching (Segal et al., 1985; Wang & Palincsar, 1989; Weinstein & Mayer, 1986). Research 
results on metacognition have been especially helpful in developing instructional strategies for children 
from educationally disadvantaged and at-risk backgrounds (Means & Knapp, 1991). Cognitive processes 
have also been identified as highly influential. Historically, cognition, whether defined as general 
intelligence, prior knowledge, or specific subject matter competence, has always been considered of prime 
importance. That estimation is confirmed by this empirical research and the evaluations of experts. 
Motivational and affective attributes are now considered cognitive constructs and play a key role in 
students' perseverance and enthusiasm for learning. Ail of these psychological attributes are essential to 
the development of independent, seif-regulated learners. Currently, many educational and psychological 
theorists conceive of learners as architects building their own knowledge structures, a conception that 
reflects the cognitive paradigm of learning now prominent in the social sciences (Gardner, 1987). 

Instructional . Instructional variables exert significant influence on school learning. In the past 
decade, research on cIassrx)om management has demonstrated the effectiveness of a variety of instmctional 
techniques and teacher behaviors in controlling classrooms and enhancing achievement (Doyle, 1986). 
Examples of classroom management techniques include the prompt and efficient handling of routine 
tasks, the minimization of distractions and interruptions, having materials ready for use, and handling 
behavior problems in a marmer that is minimally disruptive to the classroom. 

One type of classroom interaction that has been linked to student outcomes is the amount and quality 
of teacher and student academic interactions. Academic interactions promote learning by making 
students aware of subject- specific knowledge structures and then helping them develop internal 
representations of those knowledge structures. An example of a teacher and student academic interaction 
is questioning students. Teacher questioning can be characterized by the frequency of questions, 
cognitive level of questions, range of difficulty-level, the kinds of responses encouraged (extended vs. one 
word), and post-question wait time. 

A second type of classroom interaction that has also been linked to student outcomes is the frequency 
and quality of teacher and student social interactions. When teachers engage students in social 
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interactions, they can model appropriate behaviors, dissuade students from disruptive behavior, and 
establish a classroom atmosphere conducive to learning. Positive teacher and student social interactions 
contribute to students' sense of self-esteem and foster a sense of membership in the classroom and school. 
Social interactions can also include praise and corrective feedback that guide student learning. Not all 
praise and feedback is initiated by teachers, however, students can also provide feedback and praise to 
their classmates in cooperative learning situations and through peer and cross-age tutoring. 

Home Environment . The proximal variables encompassed by the home environment include not 
only the educational characteristics of the home, but also parent activities and attitudes that support student 
learning. Representative activities and attitudes include parents’ expression of interest in student school 
work, participation in school conferences, expectations for students' academic success, and ensuring 
completion of homework and school attendance. In contrast to distal variables which are more removed 
from students' day to day lives, the home is central to students' daily experience. Consequently, the home 
functions as the most salient out of school context for student learning, amplifying or diminishing the 
school’s effect on learning. 

Implications for Practice 

If practitioners and teacher educators wish to enhance school learning, they must attend to proximal 
variables such as: (1) psychological variables, especially metacognition and cognition; (2) classroom 
instruction and management, and student and teacher social and academic interactions; and (3) the home 
environment Findings from cognitive psychology, including the importance of prior knowledge, 
individual aptitudes, and metacognitive processes, should inform teaching. Students’ prior knowledge and 
level of understanding must be taken into account as teachers attempt to structure new content. Individual 
differences in psychological aptitudes such as verbal fluency, spatial reasoning, and numeracy influence 
students’ ability to perform in school. In order to develop effective lessons, teachers need to determine 
students’ levels of prior knowledge, their relevant psychological aptitudes, and their use of learning or 
metacognitive strategies. Instructional strategies like reciprocal teaching, cognitive skills instruction, and 
adaptive instructional systems incorporate the kinds of proximal psychological variables which promote 
school learning. 

Findings on the salience of classroom instructional variables should also inform teachers practice. 
Efficient classroom management enables teachers to spend more time on instruction than addressing 
discipline problems and bureaucratic tasks. The increased quantity of time for instruction is positively 
related to enhanced student achievement. Teacher and student academic interactions promote learning by 
allowing tcachere to receive more regular feedback about the effectiveness of their instruction and to tailor 
that instruction to meet the specific needs of their students. Students benefit from academic interactions 
with teachers by receiving instruction that matches their prior knowledge, addresses their misconceptions, 
and organizes knowledge in ways that are meaningful. Instructional techniques such as scaffolding 
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mediate between student's prior knowledge and new content. Just as teacher and student academic 
interactions foster learning, so do social ones. Teachers should engage in positive social interactions with 
students to minimize disruptions, to develop an orderly classroom and safe school environment, to 
encourage creativity and tolerance towards divergent points of views, and to promote the value of learning. 

Because of the imp>ortance of the home environment to school learning, teachers must also develop 
strategies to increase parent involvement in their children's academic life. This means teacher should go 
beyond traditional once-a-year parent/teacher conferences and work with parents to see that learning is 
valued in the home. Teachers should encourage parents to be involved with their children's academic 
pursuits on a day to day basis, whether helping with homework, monitoring television viewing, reading to 
their young children, and simply expressing the expectation that their children will achieve academic 
success. 

The evidence linking distal to proximal variables and to learning is sparse. Distal variables such as 
district and state policies may set the stage for classroom practices that affect student learning, but findings 
from the present review provide little supporting evidence. Distal policies are likely to make a major 
difference in learning only when they affect proximal practices. Indeed, these findings may be reflective 
of a lack of implementation and/or the complexities that are generally associated with the implementation 
of distal processes. 

Two major findings from the present review suggest important policy implications: the actions of 
students, teachers, and parents matter most to student learning; policies at the program, school, district, 
state, and federal levels have limited effect compared to the day-to-day efforts of the people who are most 
involved in students' lives. Knowing that proximal variables have a greater impact on school learning than 
distal ones, educators, when formulating policies, should be mindful of where they can make the biggest 
difference in terms of the student, the classroom, and the home. 

CONCLUSIONS 

Three huge bodies of evidence suggest that a knowledge base for school learning has been emerging 
in the last several decades. Hard-won evidence is attributable to efforts of thousands of primary 
researchers whose contributions can now be synthesized in several ways. Conventional reviews, meta- 
analyses, and expert ratings show moderate to substantial agreement on the relative sizes of influences of 
variables on academic learning. This review, however, uncovers some discrepancies among the three 
sources of information that should be incentives for additional research. It can be hoped that future 
primary studies and syntheses will produce greater consistency. 

Still, there are limitations on the ultimate precision that can be sought Even in the primary studies, 
estimates are affected by the validity of measures of the independent and dependent variables, the match 
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between what is taught and tested, the sampling of students, and other factors. In some instances, however, 
the greater the shortcomings in validity, the greater the underestimation of effects. 

Reviewers choose or are chosen to write about teaching methods and other educational variables that 
have interested them; selective conceptual biases may be operating. Reviewers may underestimate or 
overestimate effects; certainly they are affected by their own theoretical and/or disciplinaiy proclivities, 
spirit and fads of the times, and methodological limitations and advances. Psychology, for example, has 
not always preserved a balance among behavior, cognition, and conation. 

In projecting future results from the present findings, great caution is necessary. Poorly implemented 
versions of previously successful practices, especially those shown in special circumstances, are unlikely to 
work as well. Some practices that work well in some settings and with some students may not work as well 
with others, although evidence for such exceptionality is easier to hypothesize than to show consistently. 
The aggregated estimates nonetheless provide one reasonable basis for formulating educational policies 
and practices. They represent what can be distilled from an enormous body of educational research 
extending over the last half century; and the independent sources of evidence show reasonable agreement. 

Ironically, state, district, and school policies that have received the most attention in the last decade of 
educational reform appear least influential on learning. Changing such remote policies, even if they are 
well intentioned and well founded, must focus on proximal variables in order to result in improved 
practices in classrooms and homes, where learning actually takes place. 
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appendix a 

Technical Information on the Content Analyses 
of Research Literature on School Learning 

Selection of a Corous of Studies 

The final corpus of studies summarized in the content analyses included authoritative reviews 
and handbook chapters. The final corpus included journals and books published by the American 
Educational Research Association, government documents and literature recommended by the 
Scientific Advisory Board at Temple University. Chapters from the Annual Review pf PsYChplogy . 
the Annual Review of Sociology. De s igns for ComoensatPry E<lucatlQn (Williams, Richmond, & 
Mason, 1986), the Handbook of Research on Teaching (Wittrock, 1986), the Review of Researghj n 
Education, other handbooks, and journal review articles were included to ensure coverage of every 
category in the conceptual frameworic In all, over 200 chapters and review articles were identified 
and read for possible inclusion, and 179 were selected for coding. 

Coding Procedure 

The authors developed a 3-p>oint scale to code the strength of each of the 228 variables relation 
to school learning. Variables with weak relations to learning were coded 1, those with moderate 
relations were coded 2, and those with strong relations were coded 3. Variables were coded on the 
basis of statistical measures (effect sizes/correlations), quantitative measures (the proportions of 
studies confirming a variable's strength), and qualitative measures (descriptions of results). 

For those studies that repotted results in terms of effect sizes or correlations, a score of 1 was 
given if the effect size was less than .10; a score of 2 was given if the effect size was between .10 and 
.33; and a score of 3 was given if the effect size was greater than .33. For those variables for which 
quantitative measures were reported, a score of 1 was assigned if less than 40% of the reported studies 
found a statistically significant relation to a learning outcome; a score of 2 was assigned if between 
40% and 80% of the reported studies found a significant relation; and a score of 3 was reserved for 
those variables in which more than 80% of the repotted studies indicated a significant relation. For 
those items for which no statistical or quantitative indicators were reported, a judgment of weak (1), 
moderate (2), or strong (3) was made on evidence provided in the document s prose description of 

the results. 

A 15-page coding form was used to record the detailed ratings for each source (handbook 
chapters, review articles, etc.). Over 2,500 pages of coding forms containing the detailed ratings were 
completed. Both the page number in the source and the reported strength for each variable were 
recorded for each citation or discussion. Obviously, none of these sources discussed all 228 
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variables. In any given source, however, there might be multiple ratings of strength and several 
discussions or results reported on any number of the 228 variables. Because the sources coded are 
handbook chapters and review articles, information on specific features of the primary studies (such 
as the unit of analysis, grade levels, or subject matter) summarized in these chapters could not be 
coded— rather, the sjmthesis focuses on the strength of influences and effects. 

The coding process yielded about 10,000 detailed ratings, which were summarized on a 
summary form for each of the 179 sources. Recorded on the summary forms was an overall mean 
rating of strength of influence for each of the 228 variables discussed. After calculating an overall 
mean for each of the 228 variables for a given source, the 10,000 detailed ratings were aggregated 
into about 3,700 summary ratings which were then statistically analyzed. 



Data Summary 

The 3,700 summary ratings were further aggregated into 30 categories. As reported eariier, 
means, standard deviations, alpha reliabilities, and frequency of mention were calculated for these 30 
categories. (Wang, Haertel, & Walberg, 1990, p. 34). 
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APPENDIX B 

Technical Information on the Meta-Analyses of Research Literature on School Learning 

The findings in Fraser et al. (1987) were compiled from the results of many meta-analyses (or 
quantitative syntheses). Quantitative results from chapters 2, 3, and 4 were used in the current study. 
Chapters 1 and 6 were not included because they did not contain quantitative results. Chapter 5 was 
not included because it summarized the results of chapter 2. The current study used 85 meta- 
analyses discussed in Fraser et al . Their results were recorded in 23 of the 30 categories of the 
conceptual framework. Because the meta-analyses summarized in Fraser et al. did not cover all 30 
categories, the authors of this article conducted a library search described at the end of this appendix. 

Description of the Contents of Fraser et al. (19871 Chapters 2. 3. and 4 

Chapter 2, "Syntheses of Research on Factors Influencing Learning," summarized meta-analyses 
of approximately 2,575 individual studies that identified nine aptitudinal, instructional, and 
environmental factors that have consistently exhibited strong influences oh student learning (See 
Fraser et al.. p. 155, for examples of the studies included). The results spanned 50 years of research 
conducted within and outside the United States. These results included narrative reviews and 
quantitative syntheses or meta-analyses summarizing results for each of Walberg’s nine productivity 
factors. In addition, results from three large sets of statistical data on elementary and high school 
students were included. These data included information from the National Assessment of 
Educational Progress (NAEP), High School and Beyond, and the International Study of Educational 
Achievement (lEA). Also included were results from studies of the most effective ways to bring 
about constructive changes in schools as well as results from case studies of Japanese and American 
classes, which compared educational productivity in these two countries. Results from the syntheses 
of the several thousand individual studies were presented as correlations or effect sizes. 

Qiapter 3, "Contextual and Transactional Influences on Science Outcomes," presented results of 
meta-analyses of individual bivariate studies conducted within and outside the United States. This 
chapter focused exclusively on educational productivity in science education. The impact of 
contextual and transactional factors on science outcomes was synthesized. Contextual factors 
included, for example, student characteristics, teacher characteristics, curriculum materials, facilities 
and equipment, home environment, and school climate. Transactional factors included student 
behaviors, teacher behaviors, external intrusions, instructional research exposure, and classroom 
climate. The science outcomes studied included student achievement, student attitudes, student skills. 
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teacher change, scientific literacy, and career choices. The research syntheses and meta-analyses 
summarized in this chapter were based on ERICs yearly reviews of science education. Project 
Synthesis results, and new quantitative syntheses reported in the research literature. (For examples of 
the meta-analyses summarized, see Fraser et al., 1987, pp. 167-182.) Results from the syntheses were 
reported as mean correlations, mean effect sizes, or aggregate multiple regression results. 

Chapter 4, "Identifying the Salient Facets of a Model of Student Learning; A Synthesis of 
Meta-Analysis," presented results from 134 meta-analyses of achievement outcomes and 92 meta- 
analyses of attitude outcomes. The corpus of studies synthesized in this chapter was identified using 
a computer search of psychological abstracts, dissertation abstracts, and ERIC. Only research 
syntheses with 10 or more studies related to achievement were included. All results were presented as 
overall correlations. 

Library Search to Identify Additional .Syntheses 

A library search was conducted to provide coverage for the seven categories not represented in 
Fraser et al. (1987), as well as to provide results from more recent syntheses. Thirty-six new sources, 
compiled from journal articles, books, and an ERIC search, were identified as possible supplements. 
After evaluating the new sources for their type and quality of data, only 12 of the 36 syntheses 
contained quantitative data. However, because six of the studies reported in Fraser et al. also were 
reported among the results in the 12 additional research syntheses, only the other six additional 
research syntheses were added to the corpus. The six additional syntheses did not increase the 
coverage of the categories of missing data, but they did provide more comprehensive coverage of 
some of the 228 variables within the 23 categories. The final list of quantitative studies synthesized 
for the current article is presented in Appendix C. 

Description of the Decision Rules IJspd 

Because the results of chapter 4 in Fraser et al. (1987) were not limited to science education (as 
was chapter 3) and because they contained results from 134 syntheses (including some of those 
reported in chapter 2), chapter 4 became the starting point for the selection of correlations and effect 



The first decision rule was to scan chapter 4 for those results that most closely matched the 
definitions of the 30 categories. If a single result matched one of the categories, that result was 
selected. If several results were reported that matched one of the categories, the statistical average of 
those results was recorded. In chapter 4, for example, a variety of correlational results was presented 
that could be defined as cognitive variables; these results included intelligence, general ability, prior 
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achievement, Piagetian developmental level, cognitive ability, and cognitive style. The average of the 
correlations associated with these variables was computed and recorded for the cognitive category. 

For those categories where no match was found, a second decision rule was applied: Scan 
chapter 2 for those results that most closely matched the definition of the remaining categories. For 
those categories where a match was yet to be found, a third decision rule was applied: Scan chapter 3 
for those results that most closely matched the definition of the remaining categories. Finally, if after 
scanning all three chapters and the six additional syntheses identified in the library search, there was 
no match found for a particular category, the category remained empty and was assigned a missing 
data value. 

Effect Size Transformation to Correlation 

The effect sizes id) were transformed into correlations (r) using the following formula (Cohen, 

1969): 

r = d! id^ + 4) V2 

Formula and Procedure in Weighting Mean Correlations 

Because each of the mean correlations in this study was itself an aggregated statistic, it 
represented different numbers of relationships. Weighting allowed those mean correlations based on 
a large number of relationships to exert more influence in the calculation of the single, weighted 
mean correlation for the category. Mean correlations based on one or a small number of 
relationships exerted, then, less influence. For a category combining results from three syntheses, for 
example, the following formula was employed to calculate the weighted mean correlation (Mw). The 
mean correlation (r) for each synthesis was multiplied times the number of statistical relationships in) 
in the synthesis to arrive at a product These three new products were summed and then divided by 
the sum of all the statistical relationships in the three syntheses. This produced the weighted mean 
(Mw) for the category. 

Mw = E(r * n)fZin) 



O 
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APPENDIX C 



Meta-analyses of Educational Research taken from Fraser, Walberg, Welch, and Hattie (1987). 
Syntheses of Educational Productivity Research. In The International Journal of Educational 
Research, Vol. 11, No. 2, and Additional Subsequent Studies. 



Aiello, N. C., & Wolfe, L. M. (1980, April). A meta-analvsis of individualized instruction in science . 
Paper presented at the Annual Meeting of the American Educational Research Association, 
Boston. 

Anderson, R. D. (1983). A consolidation and appraisal of science meta-analyses. Journal of 
Research in Science Teaching. 20 ., 497-509. 

Ascencio, C. E. (1984). Effects of behavioral objectives on student achievement: A meta-analysis of 
findings. Dissertation Abstracts International. 45. 501 A. 

Athapilly, K., Smidchens, U., & Kofel, J. W. (1983). A computer-based meta-analysis of the effects 
of modem mathematics in comparison with traditional mathematics. Educational Evaluation and 
Policy Analysis. 5. 485-493. 

Bangert, R. L., Kulik, J. A., & Kulik, C. C. (1983). Individualized systems of instruction in secondary 
schools. Review of Educational Re.search. 53. 143-158. 

Bloom, B. S. (1976). Human characteristics and school learning . New York: McGraw-Hill. 

Boulanger, F. D. (1981). Ability and science learning: A quantitative synthesis. Journal of 
Research in Science Teaching. 18. 113-121. 

Brcdderman, T. (1983). The effects of activity-based elementary science on student outcomes: A 
quantitative synthesis. Review of Educational Research. 83. 499-518. 

Bryant, F. B. (1983, August). Desegregation and black student achievement: Results from a meta- 
gnalysis- Paper presented at the annual meeting of the American Psychological Association, 
Anaheim, CA. 

Bums, P. K., & Bozeman, W. C. (1981). Computer-assisted instruction and mathematics 
achievement: Is there a relationship? Educational Technology. 21. 37-38. 

Cohen, P. A. (1981). Student ratings of instruction and student achievement: A meta-analysis of 
multisection validity studies. Review of Educational Re.search. 51. 281-309. 

Cohen, P. A., Kulik, J. A., & Kulik, C. C. (1982). Educational outcomes of tutoring: A meta- 
analysis of findings. American Educational Research Journal. 19. 237-248. 

Crain, R. L., & Mahard, R. E. (1983). The effect of research methodology on desegregation- 
achievement studies: A meta-analysis. American Journal of Sociology. 88 . 839-854. 

Curbelo, J. (1985). Effects of problem-solving instruction on science and mathematics student 
achievement: A meta-analysis of findings (discovery learning, critical thinking, inquiry, 
inductive, scientific). Dissertation Abstracts International. 46. 23A. 

Druva, C. A., & Anderson, R. D. (1983). Science teacher characteristics by teacher behavior and by 
student outcome: A meta-analysis of research. Journal of Research in Science Teaching. 20 
467-479. 
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Dusck, J. B., & Joseph, A. (1983). The bases of teacher expectations: A meta-analysis. Journal of 
Educational Psychology. 75. 327-346. 

El-Nemr. M. A. (1980). Meta-analysis of the outcomes of teaching biology as inquiry. Pissenation 
Abstracts International. 40 . 5813A. 

Findley, M. J., & Cooper, H. M. (1983). Locus of control and academic achievement: A literature 
review. Journal of Personality and Social Psychology, 44, 419-427. 

Fleming, M. L, & Malone, M. R. (1983). The relationship of student characteristics and student 
performance in science as viewed by meta-analysis research. Journal o f Research in Science 

Frederick, W. C. (1980). Instructional time. Evaluation in Education. 4. 117-118. 

Freeman. H. E. (1984). A meta-analysis of gender differences in mathematics achievement (sex 
differences). Dissertation Abstracts International. 41. 501 A. 

Giaconia, R. M.. & Hedges, L. V. (1982). Identifying features of effective open education. Review 
of ^ucational Research. 52. 579-602. 
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Glass, G. V., McGaw, B., White, K., & Smith, M. L. (1980). Integration of research studies: Meta- 
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Institute of Education. ERIC Document Reproduction No. ED208003 
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learning: A quantitative synthesis. British Educational Research Journal. 7. 27-36. 
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Hattie, J. A., & Hansfottl, B. C. (1982, September). Personality and intelligence: What relationship 
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Figure 1 

Stem-snd-Leuf Plots of Residusis 
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Table 1 



Description of Theoretical Constructs, Categories, and Illustrative 
Variables Incorporated into the Theoretical Framework 



State and District Governance and Organization 

These categories are assc^iated with state- and district-level school governance and administration. They include 

P°/'C'es. testing and graduate requirements, teacher licensure, specific provisions in 
teacher contracts, and some distnct-level administrative and fiscal variables. 



■Category 

District Demographics 
State and District Policies 



Illustrative Variable 
School District Size 
Teacher licensure requirements 



Home and Community Educational Contexts 

^we categones are associated with the home and community contexts within which schools function. They 
1 dude conmunity demographics, peer culture, parental support and involvement, and amount of time students 
spend out of school on activities such as television viewing, leisure reading, and homework. 



Category 
Community 
Peer Group 

Home Environment and Parental Support 
Student Use of Out of School Time 



Illustrative Variable 

Socioeconomic level of community 

Level of peers’ academic aspirations 

Parental involvement in ensuring completion of homework 

Student participation in clubs and extracurricular activities 



School Demographics, Culture, Climate, Policies, and Practices 

ass^iated with school-level demographics, culture, climate, policies, and practices. They 
^lude demographics of the student body; whether the school is public or private, and levels of funding for 

progr^s. school-level decision-making variables; and specific school-level policies and 
practices, including policies on parental involvement in the school. 



Category 

School Demographics 
Teacher/Administrator Decision-Making 
School Culture (Ethos conducive to 
teaching and learning) 

Schoolwide Policy and Organization 
Accessibility 

Parental Involvement Policy 



Illustrative Variable 
Size of school 

Principal actively concerned with instructional program 
Schoolwide emphasis on recognition of academic 
achievement 

Explicit schoolwide discipline policy 
Accessibility of education program (overcoming 
architectural communication and environmental barriers) 
Parental involvement in improvement and operation of 
instructional program 



Design and Delivery of Curriculum and Instruction 

^ese categories are associated with instruction as designed and with the physical arrangements for its deliven' 
They include the instructional strategies specified by the curriculum, and characteristics of instructional materials. 



Category 

Program Demographics 
Curriculum and Instruction 
Curriculum Design 



Illustrative Variable 

Size of instructional group (whole class, small group, and one- 
on-one instruction) 

Alignment among goals, contents, instruction, 
assignments, and evaluation 
Materials employ advance organizers 



er|c 
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Table 1 (Continued) 



Classroom Practices 

These categories are associated with the implementation of the curriculum and the instructional program. They 
include classroom routines and practices, characteristics of instruction as delivered, classroom management, 
monitoring of student progress, quality and quantity of instruction provided, student/teacher interactions, and 
classroom climate. 



CatgRog 

Classroom Implementation Support 

Classroom Instruction 
Quantity of Instruction 

Classroom Assessment 

Qassroom Management 

Student and Teacher Social Interactions 

Student and Teacher Academic Interactions 

Classroom Climate 



Illu.strative Variable 

Establishing efficient classroom routines and 
communicating rules and procedures 
Use of clear and organized direct instruction 
Time on task (amount of time students are actively 
engaged in learning) 

Use of assessment as a frequent integral component of 
instruction 

Group alerting (teacher uses questioning/recitation 
strategies that maintain active participation by all students) 
Student responds positively to questions from other 
students and from teacher 

Frequent calls for extended, substantive oral/written 
response (not one-word answers) 

Cohesiveness (members of class are friends sharing 
common interests and values emphasizing cooperative goals 



Student Characteristics 

These categories are associated with individual students, including demographics, academic history, and a variety 
of social, behavioral, motivational, cognitive, and affective characteristics. 

Category Illustrative Variabk 

Student Demographics Gender and marker 

History of Educational Placement Prior grade retention 

Social and Behavioral Positive, nondisruptive behavior 

Motivational and Affective Attitude toward subject matter instructed 

Cognitive Level of specific academic knowledge in subject area 

instructed 

Metacognitive Comprehension monitoring (planning; monitoring 

effectiveness of attempted actions and outcomes of actions; 
testing, revising, and evaluating learning strategies) 
Psychomotor skills specific to area instructed 



Psychomotor 



Table 3 

Average T-Scores for each of the Six Theoretical Constructs 



Theoretical Construct 


Average T-Score 


Student Characteristics 


54.7 


Classroom Practices 


53.3 


Home and Community Educational Contexts 


51.4 


Design and Delivery of Curriculum and Instruction 


47.3 


School Demographics, Culture, Climate, Policies, and Practices 


45.1 


State and District Governance and Organization 


35.0 




Table 4 



T-Scores, Average T-Scores, and Number of Sources and Statistical Relationships 
by Category for the Content Analyses, Expert Ratings, and Meta-Analyses (Ordered from 

Greatest to Least Average T-Scores) 



Category 


Content 

Ratings 


Expert 

Ratings* 


Meta- 

Analyses 


Average 


Number of 
Sources in 
Content 
Ratings 


Number of 
Statistical 
Relationship: 
in Meta- 
Analyses 


Classroom Management 


59.5 


64.9 


70.0 


64.8 


42 


15 


Metacognitive 


60.0 


68.0 


61.1 


63.0 


76 


186 


Cognitive 


55.5 


58.1 


70.2 


61.3 


101 


825 


Home Environment and Parental 
Support 


51.9 


62.1 


61.3 


58.4 


47 


92 


Student and Teacher Social 
Interactions 


57.3 


56.1 




56.7 


44 


** 


Social and Behavioral 


55.5 


55.0 


** 


55.2 


35 


** 


Motivation and Affective 


53.3 


64.9 


46.2 


54.8 


81 


899 


Peer Group 


56.4 


56.1 


49.3 


53.9 


18 


122 


Quantity of Instruction 


57.3 


50.2 


53.7 


53.7 


69 


168 


School Culture 


49.2 


57.7 


52.8 


53.3 


49 


185 


Classroom Climate 


56.8 


54.2 


45.9 


52.3 


75 


734 


Classroom Instructional 


49.7 


59.3 


47.2 


52.1 


156 


4095 


Curriculum Design 


51.0 


51.0 


52.0 


51.3 


97 


752 


Student and Teacher Academic 
Interactions 


51.5 


41.9 


59.3 


50.9 


29 


14 


Classroom Assessment 


51.5 


52.6 


47.3 


50.4 


61 


45 


Community 


47.4 


50.6 


** 


49.0 


15 


** 


Psychomotor 


71.2 


36.3 


39.3 


48.9 


6 


637 


Tcachcr/Administrator Decision 
Making 


40.7 


56.1 


** 


48.4 


21 


** 


Curriculum and Instruction 


52.8 


44.3 


46.0 


47.7 


108 


1001 
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Table 4 (continued) 



T-Scores, Average T-Scores, and Number of Sources and Statistical Relationships 
by Category for the Content Analyses, Expert Ratings, and Meta-Analyses (Ordered from 

Greatest to Least Average T-Scores) 



Category 


Content 

Ratings 


Expert 

Ratings* 


Meta- 

Analyses 


Average 


Number of 
Sources in 
Content 
Ratings 


Number of 
Statistical 
Relationships 
in Meta- 
Analyses 


Parental Involvement Policy 


41.6 


43.1 


52.6 


45.8 


23 


1 


Classroom Implementation 
Support 


49.2 


48.6 


39.3 


45.7 


66 


27 


Student Demographics 


43.0 


41.1 


50.4 


44.8 


90 


904 


Student Use of Out of School 
Time 


53.7 


46.6 


32.6 


44.3 


17 


274 


Program Demographics 


55.1 


39.5 


33.9 


42.8 


23 


725 


School Demographics 


44.8 


36.3 


43.0 


41.4 


25 


491 


State and District Policies 


22.4 


32.8 


56.0 


37.0 


19 


22 


School Policy and Organization 


29.5 


39.1 


40.8 


36.5 


74 


120 


District Demographics 


32.2 


33.6 


** 


32.9 


14 


** 


Accessibility 


*** 


*** 


*** 


*** 


*** 


*** 


History of Educational 
Placement 


*** 


*** 


*** 


*** 


*** 


*** 



* There were 61 respondents who rated each of the 30 categories 

** Data were unavailable for these categories; in analyses, they were coded as missing data. 

*** The Accessibility and History of Educational Placement categories were removed prior to computation of the 

T-Scores. 
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The National Center on Education in the Inner Cities 



The National Center on Education in the Inner Cities (CEIC) was established on November 1, 1990 by the Temple 
University Center for Research in Human Development and Education (CRHDE) in collaboration with the University of Illinois 
at Chicago and the University of Houston. CEIC is guided by a mission to conduct a program of research and development that 
seeks to improve the capacity for education in the inner cities. 

A major premise of the work of CEIC is that the challenges facing today’s children, youth, and families stem from a 
variety of political and health pressures; their solutions are by nature complex and require long-term programs of study that apply 
knowledge and expertise from many disciplines and professions. While not forgetting for a moment the risks, complexity, and 
history of the urban plight, CEIC aims to build on the resilience and "positives" of inner-city life in a program of research and 
development that takes bold steps to address the question, "What conditions are required to cause massive improvements in the 
learning and achievement of children and youth in this nation’s inner cities?" This question provides the framework for the 
intersection of various CEIC projects/studies into a coherent program of research and development. 

Grounded in theory, research, and practical know-how, the interdisciplinary teams of CEIC researchers engage in studies 
of exemplary practices as well as primary research that includes longitudinal studies and field-based experiments. CEIC is 
organized into four programs: three research and development programs and a program for dissemination and utilization. The 
first research and development program focuses on the family as an agent in the education process; the second concentrates on 
the school and factors that foster student resilience and learning success; the third addresses the community and its relevance to 
improving educational outcomes in inner cities. The focus of the dissemination and utilization program is not only to ensure that 
CEIC’s findings are known, but also to create a crucible in which the Center’s work is shaped by feedback from the field to 
maximize its usefulness in promoting the educational success of inner-city children, youth, and families. 



CEIC Senior Associates 



Margaret C. Wang 
Director, CEIC and CRHDE 
Professor of Educational Psychology 
Temple University 



Aquiles Iglesias, Associate Director, CEIC 
Associate Professor and Chair, 
Speech-Language-Hearing 
Temple University 



Lascelies Anderson 
Professor and Director, 
Center for Urban 
Educational Research and 
Development 
University of Illinois 
at Chicago 

David Bartel t 
Associate Professor of 
Geography and Urban 
Studies and Director, 
Institute for Public 
Policy Studies 
Temple University 

William Boyd 
Professor of Education 
Pennsylvania State 
University 

Gayle Dakof 
Visiting Assistant 
Professor of 
Counseling Psychology 
Temple University 



Howard Liddle 
Professor of Counseling 
Psychology 
Temple University 



Temple University 



Geneva Haertel 

Senior Research Associate 

CRHDE 



Michael Goetz 
Associate Professor 
of Economics 
Temple University 



H. Jerome Freiberg 
Professor of Curriculum 
and Instruction 
University of Houston 



Judith Stull 

Associate Professor of 

Sociology 

La Salle University 

William Stull 

Professor and Chair, 
Department of Economics 
Temple University 

Ronald Taylor 
Associate Professor of 
Psychology 
Temple University 

Herbert J* Walberg 



Leo Rigsby 
Associate Professor 
of Sociology 
Temple University 



Hersholt C. Waxman 
Associate Dean for 
Research and Associate 
Professor of Curriculum 
and Instruction 
University of Houston 

Kenneth Wong 
Associate Professor 
Department of Education 
and Social Sciences 
University of Chicago 

William Yancey 
Professor of Sociology 
Temple University 



Maynard C. Reynolds 
Professor Emeritus of 
Educational Psychology 
University of Minnesota 



Research Professor 
of Education 
University of Illinois 
at Chicago 



Andrea Zetlin 
Associate Professor 
Special Education 
California State 
University, Los Angeles 
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